Test Assessment

Does the system under test behave correctly? This is one of the questions to be answered by the test assessment.

The test assessment follows the test execution. Both processes - test execution and test assessment - are strictly separated and completely independent of each other. First, a test case is designed and executed and all specified data is recorded. Then, the recorded data is assessed according to the check rules specified in the assesslets, see Test Assessment - Assesslets.

For each test assessment, the data from the test execution is de-jittered to the grid of the step size set in the Platform Configuration dialog. This is relevant, if the test was performed with real-time semantics, for example, when using the FUSION Platform.

You can adjust the assesslets content and rerun the test assessment on the already recorded test data. Thus, after changing the test assessment settings, you do not need to rerun the tests.

In nearly all assesslets, you can make use of the so-called TPT assessment language. This language provides many methods. All available methods are described in Assessment Functions.

Select Help|PDF|Assessment Quick Reference to obtain a first overview of the language.

The assessment perspective is recommended to perform the test assessment, see User Interface - Perspectives.

To find out for example if an assessment variable has got a value in at least one of the test cases or to determine the minimum and maximum values of an assessment variable depending on all test cases, use the global assessment, see Test Assessment - Global Assessment.

The common test assessment tasks are as follows:

Observation of limits

Limits can be assigned to signals and these signals monitored for their adherence to the limits. Crossing a limit may not the most relevant factor, but rather the duration, time and frequency of this non-conforming behavior.

Comparison to a reference signal

A signal x shall be compared to a signal x_ref. It may be important to monitor not only whether the signals differ but if the difference took place during a critical period or if the difference was particularly large.

Relative timing of events

The difference in time between the occurrence of event A, for example, light_switch == AUTO, and the occurrence of event B, for example, light_intensity < 60%, can be monitored.

Assessment time interval

The test assessment may only be relevant during a certain time interval during the test. For example, at a specific observation limit, during the comparison to a reference signal or just at the initialization phase of a system.

Qualitative statements

During test assessment, it shall be decided in which mode the system was operating at a certain time, for example, NORMAL, OVERHEAT or OFF. Since the system mode may vary over time, it is necessary to analyze the time intervals where a certain mode is valid.